|
Author
|
Topic: Polyscore
|
ebvan
Member
|
posted 10-16-2002 11:43 AM
  
Since it was mentioned in another thread...
I thought I would comment on Polyscore.
I have yet to talk to a single examiner that thought this product was worth the money. (except those who sell it)
Anecdotally it appears to lean towards a DI finding, have a narrow inconclusive band, and disagrees much too often with hand scored charts.( Even those submitted to quality control.)
I don't think that Johns Hopkins has sold us a bill of goods, but I wonder if their research data was slanted by only providing them with handpicked charts that examiners subjectively determined were representative of accurate findings. I have Polyscore on my axciton system but I only use it occasionally as an interrogational prop. I think anyone who lets a computer program make their decisions for them lacks the confidence to be effective in this profession.
I am always open to a differing opinion and would be intersted in hearing everyone's thoughts on this. ------------------
but then, that's just one man's opinion
IP: Logged |
lielabs
Moderator
|
posted 10-16-2002 04:32 PM
Ebvan,I agree,Polyscore does not encourage sole reliance on the product.
Examiner numerical scores are still the most reliable method. Axciton and polyscore analysis contradict each other sometimes so findings based on the computer scores are of no value unless they support numerical evaluations.
Polyscore is better at scoring zct single issue and poor with mgqt. IP: Logged |
J.B. McCloughan
Administrator
|
posted 10-16-2002 09:29 PM
I have yet to see a research study that suggests anything is more accurate than an 'original' examiner's numeric scoring. As I stated before in a separate thread, algorithms have been shown to outperform examiners in blind evaluation. I am unclear on what you are to suggest is a narrow band for inconclusive? I have personally spoken with Dale Olsen of John Hopkins University about his polyscore. He suggested it is more NDI friendly then traditional scoring methods. Polyscore is based on confirmed data of deceptive and truthful charts. The stored data is then compared with that of the charts you collected. If the data you collected falls with in one of the two ranges, polyscore classifies it as such. There is a nice disclaimer, found within the instructional documentation, that warns that the charts being compared to the data must conform to certain standards. A proverbial garbage in garbage out warning. An interesting study, that could easily be performed amongst examiners here, would be to take confirmed charts and compare them with polyscore for agreement. Although this study would not be 'scientific' or provide certain proof of polyscores value or lack there of, it would give a sense of how it is performing under actual field conditions.
[This message has been edited by J.B. McCloughan (edited 10-16-2002).] IP: Logged |
Toneill
Member
|
posted 10-16-2002 10:25 PM
I agree with the previous posts by examiners regarding polyscore. I always do my manual scoring then check and see what polyscore says...I tend to mark more favorable (without no great degree though) toward the benefit of the examinee and depending on how close it is I have found that the computer scoring is either on the money or pretty close, but still not accurate enough to be comfortable with. Additonal to polyscore, I utilize Quest, Identifi, and OSS..... Tony IP: Logged |
egelb
Member
|
posted 10-16-2002 10:29 PM
If you want to be further amazed you my want to try the latest version of Identifi. It scores just as the examiner does. Polyscore measures the stress for comparisons across all of the charts and compares that to each relevant. The GSR is "weighted" so there is little to compare with your hand scoring. Identifi scores to the right and left just as you do in a zone. With this technique you can see where you differ and make your own judgements. For an indepth writeup on how things work with Polycore you might want to see the NAS report which covers it in depth. By the way everyone must read the NAS report because the attorneys are reading it and asking questions. The two calls I got tday seem to indicate they are going after police agencies that do "screenings" to make assignments to specialized units. Since I feel the "utility" of such testing may justify it I turned down the retainer to attack screenings in court in accord wth the NAS findings. Be interesting to see who the attorney will use now. I keep telling my wife I should have been a country doctor.
Ed Gelb[This message has been edited by egelb (edited 10-16-2002).] IP: Logged |
Bob
Member
|
posted 10-16-2002 11:03 PM
I find Polyscore Evaluation to be very frustrating. If a hand score is NDI, and Polysore is DI, (and vice versa) there is no way to 'cross compare' in an effort to determine 'what the algorithm is seeing' which the examiner is not.As indicated in the documentation (and personal conversation with John Harris) large reactivities in the Spot/Vertical Score does not necessarily contribute to the Overall Evalution of DI/NDI- but should be used to 'rank RQ's'to which question was most problematic for the subject. Although I've noticed that a Spot/ Vertical reactivity value".65 or more" regardless of Polyscore's Overall NDI,- that my Hand score will most likely be DI/INC, although not necessarily to the same RQ. By the way, just for info- during an email with Harris, I asked what was meant in Polyscore's documentation that BP cuff"should not be beyond diastolic pressure." I found the statement confusing, as I began to wonder about a person having, say a BP of120/80. If a cuff pressure of 60mg is used, it could be interpreted as being beyond diastolic (hence no Korotkoff sounds)- yet on the other hand I couldn't imagine they were suggesting 'above' diastolic. Anyway, the statement does mean Under diastolic,and he recommends pressures of 60-65mg as that is what the algorithm research was based on. It was unknown if higher cuff pressures were used, what effect, if any, it would have on the validity/reliabilty of the program. IP: Logged |
ebvan
Member
|
posted 10-17-2002 08:09 AM
What I meant by referring to an apparent narrow inconclusive band in Polyscore is that on the surface it rarely calls a chart "Inconclusive". I have seen it take charts containing what appears to be substandard data and render a D.I. decision.
Not being qualified to understand the mathmatics and inner workings behind the rendering of these decisions, I am suspicious concerning what makes it tick. Especially since it often disagrees with scoring by experienced examiners. If one can't trust it every time, it seems to have little value. I have looked at some of the other scoring software and identifi does look good. I know that it looks like it scores a chart just like an examiner does, but how do we know that is not just an appearance issue. I haven't used the program so I just don't know.
I have been reading about OSS and what I really like about it is that the examiner measures and enters the data. I can easily look at and recognize the source for the information that is entered into the calculation process. I haven't switched over yet but it looks promissing. I'm still reading. I may soon start using it adjunct to my hand scoring to see how it compares. IP: Logged |
polyops
Member
|
posted 10-17-2002 10:45 AM
I agree with Ed Gelb that everyone should read the NAS report. Like it or not, this report is going to be the defining document for polygraph validity for a long time to come. I fear its effects are going to be much worse than the OTA report in the long run. I'll post a new thread later with some ideas on how we might confront this. I'm glad to see that the APA has started with a press release on their website. But its not enough.IP: Logged |
Robin Doorn
Member
|
posted 11-05-2002 07:16 AM
Just a comment on the Polyscore. I have found it useful as a part of the scoring process, but not as an end result. I think that by looking at the values that it places on individual questions and also by looking at the weighting that it gives the different channels, that some useful information can be pulled from the program. And while I have confidence in my hand scoring, a different overall opinion on polyscore than what I reach, still makes me go back over and verify my scoring on the charts. I guess if the scoring was exactly like my hand scoring on every occasion, it wouldn't do me much good either. I do like the feature that rank orders for post test interviews, especially on subjects who are undergoing regular monitoring. Just some quick thoughts on the polyscore. IP: Logged |
nonameone
Member
|
posted 01-20-2004 02:15 PM
I have had good success with polyscaore so far in conjunction with zct exams. For MGQTs I like to compare polyscore's rank scoring with my conclusions. After about 30 exams using polyscaore I have only seen one disageement with my manual scoring and even that was not by much. IP: Logged |
ebvan
Member
|
posted 01-20-2004 03:06 PM
nonameoneI don't know which version of polyscore you are using mine is several years old. I have however; noticed that some examiners treat scoring algorythms like some detectives treat polygraph examiners.
When we tell the investigator what he wants to hear, we are "good old SOBs". When we don't we're "no good SOBs". Either way most examiners have to get used to the idea that we're a bunch of SOBs. When something or someone agrees with you you are reluctant to objectively question it's validity and reliability. When something disagrees it's much easier to question those qualities. The question you might ask yourself is this one. If you scored an examinee NDI and Polyscore scored him DI and after you went back over your charts again to make sure you didn't miss something you still disagreed with Polyscore; what would you tell the submitting official and the examninee? If you can't believe on an algorythm when it disagrees with you; how can you presume to believe it just because it calls the charts the same way you do? My contention is that if you go with polyscore, you are not fullfilling your responsibility as an examiner and if you are going with your own scoring then you don't really need polyscore to help you make up your mind. I suggest that we should only use scoring algorytms for 2 reasons
#1 as kind of an alert system to see if we should go back and take a 3rd look at our charts before we anounce our opinion.
#2 an interrogational prop to display to a DI examinee to aid in obtaining admissions.
Most products are too expensive for this limited use. I don't know of a single algorythm vendor who says that you should make calls based on their product. but then that's just one man's opinion IP: Logged |
nonameone
Member
|
posted 01-21-2004 07:37 AM
ebvanI didnt make myself clear. I dont rely on polyscore for a final dteremination. I should have said that I compare out of curiosity.I rely on my on scoring. IP: Logged |
Polygraph Place Bulletin Board
